Template Based Processor by ianjosephwilson · Pull Request #105 · t-strings/tdom

ianjosephwilson · 2026-02-17T06:43:09Z

Rewrite Processor To Be Template Based
- Enumerate interpolation cases to simplify logic and establish framework to hone correctness, specially around text types.
Move Node Processor
- Create submodule nodes/
  - Move Node processor into nodes/processor.py
  - Move nodes.py into nodes/nodes.py
- Sync tests between the node processor and the str processor using parameterized tests.
Change Component Signature(s)

def FunctionComponent(children: Template, **kwargs) -> Template:
    return t""
def FactoryComponent(children: Template, **kwargs) -> Callable[[], Template]:
    def ComponentCallable() -> Template:
        return t""
    return ComponentCallable

Change Component Children Processing Order
- Children is a Template so the actual contents are not processed until after the component is invoked.
Change Content Interpolation Rules
- Zero-arg functions no longer have special treatment:
  - to_html(t'<div>{(lambda: "dynamic")}</div>') != '<div>dynamic</div>'
  - A new formatter named callback can be used which then interpolates the return value
    - to_html(t'<div>{(lambda: "dynamic"):callback}</div>') == '<div>dynamic</div>'
- Boolean values no longer have special treatment:
  - to_html(t'<div>{False}</div>') == "<div>False</div>"
  - None is still ignored: to_html(t'<div>{None}</div>') == "<div></div>"

AhnafCodes · 2026-02-17T16:20:35Z

Move Node processor into nodes.py

Current state: nodes.py contains only data types. processor.py contains all resolution logic.

Current separation is clean — nodes.py is "output types" and processor.py is "template resolution." Moving Node processing logic into nodes.py would blur this(eventually if not immediate)
Currently processor.py → nodes.py one-way dependency is cleaner.

AhnafCodes · 2026-02-17T17:50:39Z

Why change API signature, html -> to_html ?

i would not mind html -> H atleast terse
H(t'<div>{count}</div>'), but why?

name collisions? but, then one can always do import html as to_html

AhnafCodes · 2026-02-17T23:24:07Z

to_html(t'<div>{False}</div>') == "<div>False</div>"

this breaks {condition and template} or {condition and some_func() } which pretty common used in templating.

…on non-none returns.

…d some missing annotations.

ianjosephwilson · 2026-02-23T00:30:28Z

@davepeck @pauleveritt Can you guys take another look at this when you get a chance? It is a very large conceptual change RE: the component signature and the default processor.

I walked back the more ambitious pre-compiled template concept and now both processors are using TNode directly during processing. I have both implementations lined up with each other, tdom/processor.py:to_html and tdom/nodes/processor.py:to_node. They are using shared tests.

The interpolation value options are more strict because I was trying to makes the rules more well defined.

pauleveritt · 2026-02-23T00:38:10Z

Thanks for all the work @ianjosephwilson I'll see if @davepeck has time this week to go through it together. (Dave: Any afternoon/evening my time is fine.)

davepeck · 2026-02-24T17:45:50Z

@ianjosephwilson Thank you for this -- I think I have time this week to re-focus energies and read over in more detail. How does this PR relate to the other open and draft PRs we currently have?

(@AhnafCodes -- thank you for hopping in here, and nice to have you on board! I agree that {condition and "foo"} is something we want to support. It's a very common idiom in JSX-land and, when in doubt, I think we should draw our intuitions from there.)

ianjosephwilson · 2026-02-24T19:37:54Z

@ianjosephwilson Thank you for this -- I think I have time this week to re-focus energies and read over in more detail. How does this PR relate to the other open and draft PRs we currently have?

@davepeck At this point it might be easier to just browse the processor.py and processor_test.py files instead of diffs to get a feel for what changed. I think these are the major things:

---	Iterative Processor?	Precompilation?	TNodes parsed?	Internally uses Node?	Template Component Signature	Reduced Dynamic Intepretation
This PR	Yes	No	Yes	No	Yes	Yes
Transformer POC (closed)	Yes	Yes	Yes	No	Yes	Yes
Main	No	No	Yes	Yes	No	No

Iterative Processor
- Uses a stack/queue with a loop to process the templates and writes the html strings out as it goes. Usually this uses less memory, keeps the internals internal and doesn't have issues with the function stack limit (although that would probably be unlikely).
Precompilation
- Instead of walking the TNode tree every processing call and dynamically dispatching an interpolation "handler" the handlers were mostly pre-determined and stored in a "precompiled" Template. This "flattened" the processing step into simply walking a Template instead of traversing the TNode tree. You foresaw this as being too complicated and I think the SVG issue was really the straw that broke the camels back, at least for now.
Internally use Nodes
- The prior Node processor would create a tree of Nodes and then "stringify" the nodes into a string.
Template Component Signature
- Simplified component interface where children and output are instances of Template
Reduced Dynamic Intepretation
- Component output is more like a "tag" and less like a "primitive value"
- Tried to reduce the number of checks per interpolation (for performance, api surface area, debugging, verification, etc.)
  - No special handling for booleans
  - No special handling for callables
  - No special handling for Node

ianjosephwilson · 2026-02-25T22:25:17Z

@davepeck Would it be easier if I broke some of the smaller less related pieces out of this and we merged those first?

davepeck

@davepeck Would it be easier if I broke some of the smaller less related pieces out of this and we merged those first?

I'm sort of working through it in one fell swoop right now. Thanks for all the guide posts here. I do think there are some separable pieces we could consider teasing apart into separate PRs, but I dunno... that may be more work than it's worth.

ianjosephwilson added 24 commits February 9, 2026 22:23

Add helper method to parallel Template.__iter__.

ab74226

Move html dunder protocol into module and add tests.

0c76fc7

Split out component kwargs resolution for testing/reusability.

da0337c

Rename html to to_node and introduce to_html for string serialization.

a661404

Conditionally allow Markup for tags with non-normal text.

a412e6f

First draft using transformed templates.

8204698

Move html spec info into utility module.

4f9fd5c

Add shorthand for pulling interpolations by index back into a template.

9fcd2b7

Move node builder into nodes.

a5a5d06

Remove extra arg to escaping.

fd18de6

Unescaped entities need to be re-escaped.

678755b

Coerce tests into two processor strategies.

c233581

Parsed entities should be re-escaped by default

2c7f718

Formatting.

df2854f

Use runtime parent tag when component has no predefined parent tag.

6036697

Split assertions and correct the expected output.

96f4654

Add callback as explicit opt-in formatter.

027ddd5

Merge extra transform processor tests into main processor tests file.

bcd5b74

Remove tests that no longer apply.

55af193

Group very special case tests together with class.

79955e5

Do not call str() prematurely to allow __html__ through.

32fd5ca

Explicitly allow __html__ from exact interpolations in non-normal texts.

2f9f88f

Safe is not needed in test anymore.

e8be5cf

Use protocol

4fcfb0b

Draft direct str processor.

280c432

ianjosephwilson added 2 commits February 17, 2026 15:49

Skip attrs resolution if there are no attrs.

0179537

Skip attrs resolution if there are no attrs.

68c7d73

ianjosephwilson added 23 commits February 21, 2026 01:01

Properly handle None in text resolved without recursion.

e3a8915

Regroup element tests by content type.

9b7969c

Correctly handle None for resolve without recursion texts in to_node.

326c6cb

Move placeholder collision test from processor to parser tests.

b1a2a42

Format.

2bda54c

Start to unify testing between str processor and node processor.

b65eb07

Replace duplicate tests with basic integration test.

8086295

Drop stream naming in favor of process, fixup return types, only act …

62b60cd

…on non-none returns.

Context changes inside method so don't alter it before.

6a309ba

Realign node processor with str processor.

3603175

Actually utilize TemplateRef.

49af615

Add test for entity re-escaping in literals in normal test.

03222e7

Fix import and type annotation.

438e303

Apply format_spec to raw text and escapable raw text interpolations.

c1cdcd7

Test format_spec and conversion for escapable raw text and raw text.

99d2301

We did this.

422fe8f

Restore processor cache vs nocache test.

1324c6d

Delegate caching/parsing to an injected service.

3bd76e2

Update cache test to use new injected service.

c3440fc

Format

ba58701

Fix import.

90c6040

Add instance check for clarity and ... type check.

8cb5161

Add casts to trigger type checking of important value transitions, ad…

8e78111

…d some missing annotations.

ianjosephwilson marked this pull request as ready for review February 23, 2026 00:30

davepeck reviewed Feb 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Template Based Processor#105

Template Based Processor#105
ianjosephwilson wants to merge 59 commits intot-strings:mainfrom
ianjosephwilson:ian/transformer_rebuild

ianjosephwilson commented Feb 17, 2026 •

edited

Loading

Uh oh!

AhnafCodes commented Feb 17, 2026

Uh oh!

AhnafCodes commented Feb 17, 2026 •

edited

Loading

Uh oh!

AhnafCodes commented Feb 17, 2026 •

edited

Loading

Uh oh!

ianjosephwilson commented Feb 23, 2026

Uh oh!

pauleveritt commented Feb 23, 2026

Uh oh!

davepeck commented Feb 24, 2026 •

edited

Loading

Uh oh!

ianjosephwilson commented Feb 24, 2026

Uh oh!

ianjosephwilson commented Feb 25, 2026

Uh oh!

davepeck left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ianjosephwilson commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AhnafCodes commented Feb 17, 2026

Uh oh!

AhnafCodes commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AhnafCodes commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianjosephwilson commented Feb 23, 2026

Uh oh!

pauleveritt commented Feb 23, 2026

Uh oh!

davepeck commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ianjosephwilson commented Feb 24, 2026

Uh oh!

ianjosephwilson commented Feb 25, 2026

Uh oh!

davepeck left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ianjosephwilson commented Feb 17, 2026 •

edited

Loading

AhnafCodes commented Feb 17, 2026 •

edited

Loading

AhnafCodes commented Feb 17, 2026 •

edited

Loading

davepeck commented Feb 24, 2026 •

edited

Loading